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1  Statement  of  the  Problem 

In  the  first  part  of  this  project,  we  study  several  stochastic  control  problems  driven  by  one¬ 
dimensional  fractional  Brownian  motion(fBM)  process.  Such  processes  are  non-Markovian 
and  their  future  behavior  may  depend  on  the  history.  These  models  arise  in  many  practical 
situations  and  our  work  is  motivated  by  approximating  large  on-off  queueing  networks  by 
fBM  driven  stochastic  systems.  We  address  several  stochastic  optimal  control  problems  for 
fBM  driven  models.  We  obtain  existence  theorems  for  optimal  strategies.  In  the  case  of  long- 
run  average  cost  optimization  problems,  due  to  highly  non-Markovian  nature  of  the  fBM,  it 
is  not  apriori  known  whether  the  initial  data  plays  a  role  in  the  overall  cost  and  hence  for 
the  choice  of  an  optimal  strategy.  We  answer  this  problem  by  showing  that  the  long-run 
cost  is  independent  of  the  initial  data  and  the  tool  we  developed  here  to  obtain  this  result  is 
a  coupling  time  argument  and  it  may  have  other  applications.  Stochastic  control  of  models 
driven  by  fBM  is  a  new  research  area  and  our  contributions  will  help  further  development  of 
this  area  and  also  to  understand  the  behavior  of  controlled  large  on-off  queueing  systems.  It 
is  well  known  that  stochastic  comparison  theorems  are  very  useful  in  the  analysis  of  control 
problems  driven  by  the  ordinary  Brownian  motion.  Here  we  develop  such  comparison  results 
for  models  driven  by  fBM  and  they  can  be  used  in  finding  optimal  strategies  for  stochastic 
control  problems. 

In  the  second  part,  we  address  several  control  problems  related  to  stochastic  networks  in 
heavy  traffic,  such  as  internet  traffic,  telephone  call  centers  with  large  number  of  servers  and 
large  processing  networks.  In  a  queueing  system  with  a  finite  waiting  room  capacity  in  heavy 
traffic,  customers  tend  to  abandon  the  queue  if  the  delay  is  too  long.  On  the  other  hand, 
if  the  waiting  room  is  too  small,  it  will  be  full  most  of  the  time  and  customers  are  rejected 
often.  Thus  a  system  manager  will  face  the  problem  of  appropriate  waiting  room  size  and 
it  is  preffered  to  choose  this  waiting  room  capacity  in  a  dynamic,  history-dependent  fashion 
leading  to  non-Markovian  strategies.  We  formulate  this  as  an  optimal  control  problem  and 
derive  a  threshold  type  optimal  strategy.  We  also  investigate  the  behavior  of  offered  waiting 
times  for  impatient  customers  in  a  queueing  system  in  heavy  traffic  and  establish  a  new  class 
of  diffusion  approximations. 

*  Research  supported  by  US  Army  Research  Offce  grant  W911NF0710424. 
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2  Controlled  queueing  networks  with  fractional  Brow¬ 
nian  motion  input. 

In  a  recently  published  article  [1],  we  have  analyzed  the  control  of  a  single  server  queueing 
model  with  a  fractional  Brownian  motion(fBM)  input.  I  was  the  main  author  of  this  article. 
In  this  model  ,  for  a  given  initial  value  x  >  0  and  a  fixed  control  variable  u  >  0,  the  controlled 
state  process  Xx  is  defined  by 

X2(t)=x-ut  +  a(u)WH(t)  +  L*(t)t  t>  0,  (2.1) 

where  the  process  L“  is  given  by 

Lx(t)  =  —  min{0,  min  (x  —  us  +  <j(u)Wh(s))  },  t>  0.  (2.2) 

se[o,t] v 

From  the  above  equations,  it  follows  that  X i“(t)  >  0  for  all  t  >  0.  Notice  that  the  process  L“ 
has  continuous  paths,  and  it  increases  at  times  when  Xx(t)  =  0.  The  processes  Xx  and 
ar  both  adapted  to  the  fBM  hltration.  The  function  o  is  continuous  and  takes  only  positive 
values.  The  process  Xx  can  be  thought  of  as  the  queue-length  of  a  single-server  queueing 
system  with  a  fBM  input  and  in  such  a  situation,  the  control  variable  u  is  related  to  the 
deterministic  service  rate. 

Our  work  is  motivated  by  the  control  problems  associated  with  a  discrete  queueing  network 
model  related  to  internet  traffic  data(ON-OFF  processes).  Such  discrete  control  problems 
are  not  easily  tractable.  But,  under  heavy  traffic  conditions  as  describe  in  [1],  suitably  scaled 
queue  length  process  converges  in  distribution  to  the  above  model  (2.1).  This  can  be  verified 
using  the  fundamental  theorem  for  ON-OFF  processes  as  given  in  [5].  For  details,  see  [1]. 
Therefore,  the  analysis  of  the  control  problems  related  to  our  model  enables  us  to  understand 
asymptotically  optimal  strategies  for  the  corresponding  controlled  discrete  queueing  networks 
in  heavy  traffic.  We  have  analyzed  three  types  of  control  problems  related  to  this  model  as 
described  below: 

A.  Ergodic  Control  problem. 

We  consider  the  minimization  of  the  long  run  average  cost  functional 

I(u,x)  :=  limsup  ^=e(  f  [h(u)  +  C(X%(t))]dt+  [  pdLux(t)\ 

T—>oc  -L  Vo  J  0 

(2.3) 

where  h  and  C  are  non-negative  cost  functions  defined  on  [0,  oo)  and  p  is  a  non-negative 
constant.  These  cost  functions  are  both  continuous  and  increasing  to  infinity.  The  func¬ 
tion  h  represents  the  control  cost  and  C  represents  the  congestion  cost  associated  with 
the  system.  The  constant  p  >  0  is  the  cost  per  unit  time  when  the  workload  in  the 
system  is  empty.  The  value  function  is  defined  by 


Vo(x)  =  inf  I(u,x). 

u>0 

In  [1],  we  have  obtained  the  following  results: 


(2.4) 
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Theorem  2.1. 


(%)  The  functional  I(u,x )  in  (2.3)  is  independent  of  the  initial  point  x  and  has  the 
representation  I(u)  =  I(u,x)  =  h(u)  +  pu  +  G(u). 

Here  the  function  G(u )  is  given  by  G(u)  =  E[C(ZU)\  where  Zu  :=  maxs>0{M///(s)— ms}. 

(ii)  The  value  function  V0(x)  is  independent  of  x  and  there  is  an  optimal  control  u*  >  0 
such  that  I(u*)  =  Vq(x)  for  all  x. 

Proof  of  this  theorem  involves  a  coupling  argument  of  the  above  state  process  Xf  with 
a  stationary  process,  and  we  believe  that  our  coupling  method  will  be  useful  in  the 
analysis  of  the  problems  related  to  the  long  term  behavior  of  Xf. 

B.  A  constrained  minimization  problem. 

We  can  apply  our  results  in  the  previous  theorem  to  obtain  an  optimal  control  for  the 
following  constrained  minimization  problem  driven  by  a  fBM  model.  We  are  not  aware 
of  any  other  constrained  optimization  problems  for  models  driven  by  fBM  available  in 
the  literature.  Our  model  here  is  of  the  form 

Yu{t)  =  x  -  ut  +  a(u)WH(t)  +  Ku{t).  (2.5) 

We  keep  the  initial  point  x  fixed  throughout.  Here  o  is  a  non-negative  continuous  func¬ 
tion,  /!“(•)  is  a  non-negative  non-decreasing  right-continuous  with  left  limits  (ROLL) 
process  adapted  to  the  natural  filtration  (lFt)t> 0,  where  Et  is  the  a-algebra  generated 
by  {Wh(s)  :  0  <  s  <  t}  augmented  with  all  the  null  sets.  Furthermore,  Ku(ff)  =  0 
and  the  process  Ku  is  chosen  by  the  controller  in  such  a  way  that  the  state  process  Yu 
is  constrained  to  non-negative  reals.  In  this  situation,  the  controller  is  equipped  with 
two  controls:  the  choice  of  u  >  0  and  the  choice  of  Ku  process  subject  non-negativity 
of  the  Yu  process.  The  initial  point  x  is  fixed. 

Let  m  >  0  be  any  fixed  positive  constant.  The  constrained  minimization  problem  is 
the  following: 


Minimize 

limsup  ~e([  [h(u)  +  C (Yu(t))~\idt]  (2.6) 

T— >oo  T  \J  q  / 

Subject  to: 

E(KU(T ))  /  N 

limsup - — - -  <  m.  (2.7) 

T— >oo  T 

We  have  the  following  result: 

Theorem  2.2.  Let  u*Q  be  the  optimal  control  obtained  in  Theorem  1.1  corresponding 
to  the  case  p  =  0.  Introduce  u*(m )  by  u*(m)  =  min{wQ,m}. 

Then  for  each  m>  0  fixed,  the  optimal  state  process  is  given  by  X*(t)  =  x  —  u*(m)t  + 
a(u)WH(t)  +  L*m{t)  where  L*m  is  the  corresponding  “local-time  process”  associated  with 
X*  as  similar  to  (2.1).  Hence,  the  pair  ( u*(m),L*n )  is  an  optimal  strategy  and  it  is 
independent  of  the  initial  point  x. 
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C.  Abilian  limit  relationships. 

In  relation  with  the  cost  functional  in  (2.3),  we  also  introduce  the  corresponding  finite- 
horizon  cost  minimization  problem  on  an  interval [0,  T]  and  the  infinite-horizon  dis¬ 
counted  cost  minimization  problem.  Let  the  value  functions  of  these  two  problems  be 
given  by  V (. x ,  T )  and  Va(x)  respectively.  Here  a  >  0  is  the  discount  factor  in  the  infinite 
horizon  discounted  cost  minimization  problem.  We  obtain  the  existence  of  an  optimal 
control  for  each  of  this  problems  and  establish  the  Abelian  limit  relationships  among 
the  value  functions  V0(x),  Va(x)  and  V(x,T)  as  described  in  the  following  theorem. 

Theorem  2.3.  lim  aVa(x)  =  lim  '  {\'1  ]  =  V0(x)  for  all  x. 

a— >0+  T— >oo 

In  the  proof  of  this  theorem,  we  also  show  that  the  convergence  of  the  optimal  controls 
of  Va (x) (similarly,  V(x,T ))  as  a  tends  to  zero  (as  T  tends  to  infinity),  to  the  optimal 
control  of  Vq{x). 

F.  Tandem  Fluid  Networks. 

In  a  recently  completed  article  [3] ,  we  have  generalized  the  results  of  [1]  to  tandem  fluid 
networks  in  heavy  traffic.  We  consider  a  controlled  two  station  tandem  fluid  network  with 
heavy  tailed  on-off  sources.  Under  appropriate  heavy  traffic  conditions,  we  establish  the 
weak  convergence  of  the  two-dimensional  workload  process  to  a  controlled  queueing  model 
driven  by  a  two  dimensional  fBm  input.  We  also  allow  the  components  of  the  input  fBm 
to  be  correlated  with  a  constant  correlation  coefficient.  For  this  model,  first  we  show  the 
existence  of  a  stationary  distribution  for  the  work  load  process  with  zero  initial  data.  Then, 
we  show  that  there  exists  a  stationary  work  load  process  on  the  same  probability  space  so 
that  it’s  probability  distribution  is  invariant  over  time.  Then  finally,  under  mild  assumptions 
on  the  initial  data,  we  show  that  any  state  process  coalesce  with  the  stationary  state  process 
and  the  corresponding  coupling  time  is  finite  and  has  finite  moments.  We  also  identify  the 
asymptotics  of  the  tail  of  this  stationary  distribution.  As  an  application  of  this  result,  we 
address  a  cost  minimization  problem  for  long-run  average  cost  and  show  the  existence  of  an 
optimal  strategy  which  is  independent  of  the  initial  data.  In  conclusion,  we  showed  how  to 
extend  these  results  to  a  tandem  queue  with  any  finite  number  of  stations  and  also  obtained 
the  corresponding  stationary  distribution. 

3  Stochastic  Comparison  Theorems. 

Stochastic  comparison  theorems  are  well  known  for  the  stochastic  differential 
equations(SDE’s)  driven  by  ordinary  Brownian  motion.  They  have  a  wide  range  of  applica¬ 
tions  in  stochastic  analysis  and  in  particular,  they  play  an  important  role  in  stochastic  control 
theory.  Such  results  are  used  to  compare  expected  pay-off  values  from  different  admissible 
control  strategies.  In  the  case  of  SDE’s  driven  by  fBM,  we  have  obtained  the  following  results 
related  to  stochastic  comparison  theorems  in  a  recent  article ([6]).  One  should  note  that  the 
stochastic  integral  in  the  SDE  below  is  defined  as  a  pathwise  Riemann-Stieltjes  integral  and 
this  SDE  has  a  path  wise  unique  solution  under  the  assumptions  listed  below. 
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Theorem  3.1.  Let  H  T  >  0  and  Wh  be  a  fBM  with  Hurst  parameter  H .  Consider 
two  solutions  Xt  (i  =  1,2)  to  the  SDE’s  driven  by  the  same  fBM,  where 

Xi(t)  =  Xi  +  [  bi(Xi(s))ds  +  [  a(Xi(s))dWH(s)  (3.1) 

Jo  Jo 

for  all  0  <  t  <  T.  For  i  =  1,2,  we  assume  that  the  functions  b,  and  a  are  Lipschitz 
continuous  and  the  derivative  of  a  is  locally  Holder  continuous,  and  b±(x)  >  b2(x )  for  all  x. 
If  the  initial  conditions  satisfy  x\  >  x2,  then  X\ (T)  >  X2(T)  a.s.  for  all  T  >  0. 

Next  we  prove  the  following  more  general  version  which  has  a  control  theoretic  inter¬ 
pretation.  Let  b  be  a  given  Lipschitz  continuous  function.  We  consider  a  controlled  state 
process  Xf  which  satisfies 

Xf(t)  =  x+  f  u(s)ds  + 

Jo 

where  a  is  a  Lipschitz  continuous  function  with  Lipschitz  constant  K  >  0  and  its  derivative 
is  locally  Holder  continuous.  Here  Bh  is  a  fractional  Brownian  motion  with  Hurst  parameter 
H  >  I  and  is  adapted  to  a  filtration  (JFt).  The  control  process  u  is  adapted  to  the  filtration 
(Ft)  and  f*  |w(s)|ds  <  oo  for  all  t  >  0  a.s.  We  say  u  is  an  admissible  control  process,  if  it 
satisfies  the  condition 

u(t)  <  b(Xf(t))  for  all  t  >  0  (3.3) 

and  the  corresponding  state  process  Xf  belongs  to  the  function  space  [0,  T]  for  some 

ao  >  a  >  1  —  H.  Here  ao  =  min{|,  ypj},  where  5  >  0.  If  u  is  an  admissible  control  then  we 
say  Xf  is  an  admissible  state  process.  Let  Ax  be  the  collection  of  all  such  admissible  state 
processes  Xf  described  above.  Next  we  introduce  a  process  Yx  which  is  driven  by  feed-back 
type  control  b(Yx(s))  and  it  satisfies 

Yx(t)=x+  [  b(Yx(s))ds  +  [  a(Yx(s))dBH(s).  (3.4) 

Jo  Jo 

Then,  we  have  the  following  theorem: 

Theorem  3.2.  Assume  that  the  functions  b  and  a  are  Lipschitz  continuous  and  the  derivative 
of  a  is  locally  Holder  continuous  as  in  the  previous  theorem.  Then  there  is  an  optimal  process 
Yx  in  Ax  such  that 

Yx(t)  >  Xf(t)  a.  s. 

for  all  t  >  0,  where  Xx  is  any  process  in  Ax.  Furthermore,  Yx  satisfies  the  SDE 

Yx(t )  =x+  f  b(Yx(s))ds  +  f  <j{Yx(s))dB  H(s) .  (3.5) 

Jo  Jo 

Hence  the  optimal  control  u*  is  feed-back  type  and  is  described  by  u*(t )  =  b(Yx(t ))  for  all 
t  >  0. 


a(X«(s))dB„(s) 


(3,2) 
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Both  of  the  above  theorems  are  path-wise  comparison  results.  We  have  also  obtained  the 
following  mean  comparison  theorem  when  the  Hurst  parameter  0  <  H  <  1. 

Let  b  be  a  convex,  Lipschitz  continuous  function.  We  consider  a  general  state  process  X 
which  satisfies  the  equation 


X  (t )  —  x  + 


u(s)ds  +  crBH(t). 


(3.6) 


JO 

where  a  is  a  constant,  Bh  is  a  fBM  with  Hurst  parameter  0  <  H  <  1  and  is  adapted  to  a 
filtration  (jFf).  The  control  process  u  is  also  adapted  to  the  filtration  (jFf),  |u(s)|ds  <  oo 
for  each  t  >  0  a.s.  and  it  also  satisfies 


u(t)  <  b(X(t)),  (3.7) 

for  all  t  >  0.  Next, we  introduce  the  process  Y ,  which  is  a  solution  to  the  SDE  given  by 

Y(t)  =  y  +  f  b(Y (s))ds  +  pWH(t).  (3.8) 

Jo 

where  p  is  a  positive  constant,  Wh  is  a  fBM  with  the  same  Hurst  parameter  0  <  H  <  1  in 
a  (possibly  a  different)  probability  space  and  Wh  is  adapted  to  a  filtration  (J-t)- 

The  Lipschitz  continuity  of  b  guarantees  a  strong  solution  to  (3.8)  with  respect  to  any  fBM 
Wh-  Then  we  have  the  following  theorem: 

Theorem  3.3.  Let  a  and  p  >  0  be  constants  which  satisfy  |cr|  <  p  and  the  initial  conditions 
satisfy  x  <  y.  Then  for  any  T  >  0  and  for  any  convex  function  h, 

E[h(X(T))]  <  E{h(Y(T))}. 

In  particular,  MmaxATs)!  <  if  [max  T(s)l. 

[0  ,T]  [0  ,T] 

All  the  above  theorems  are  proved  in  ([6]). 


4  Controlling  a  process  to  a  goal. 

A.  Goal  problems  without  fuel. 

We  have  solved  the  following  probability  maximization  problem  in  [6]  for  processes 
driven  by  fBM. 

Consider  a  state  process  Xf  given  by 

Xf(t)=x+  I ‘  u(s)ds  +  a BH(t)  (4.1) 

Jo 

where  a  >  0  is  a  constant.  Here  Bh  is  a  fractional  Brownian  motion  with  Hurst  param¬ 
eter  0  <  H  <  1  and  is  adapted  to  a  filtration  (JAt).  The  control  process  u  is  adapted 
to  the  filtration  (JAf),  fQ  |u(s)|(is  <  oo  and  u{t)  belongs  to  A(Xf(t))  for  all  t  >  0  a.s. 
where  each  control  set  A(y)  is  a  subset  of  R  and  the  collection  { A(y )  :  y  in  R}  is  a 
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priori  available  to  the  controller. 

Let  a  and  b  be  constants  so  that  a  <  x  <  b.  We  would  like  to  consider  the  problem  of 
choosing  the  constant  cr  and  the  drift  control  u  to  maximize  the  probability  that  the 
state  process  Xf  reach  the  goal  b  before  reaching  a.  In  the  case  of  the  models  driven 
by  standard  Brownian  motion,  solutions  to  such  problems  are  already  known. 

Our  main  assumption  here  is  that  the  existence  of  a  positive  constant  p  >  0  and  a 
Lipschitz  continuous  function  b  defined  on  R  so  that 


b(y)=  sup  A(y) 


(4.2) 


and  p  >  0  satisfies  (4.3)  below.  Admissibility  of  the  control  process  u  guarantees  that 
u(t)  <  b(Xf(t))  for  all  t  >  0. 

We  address  this  problem  in  the  following  two  cases: 

(i)  If  b(y)  >  0  for  all  y,  then  we  assume  that  available  cr  are  bounded  below  by  a 
positive  constant  and  we  let  p  =  inf  a. 

(ii)  If  b(y)  <  0  for  all  y,  then  we  assume  that  the  available  cr  are  bounded  above  and 
in  that  case,  we  set  p  =  sup  cr. 


Therefore,  in  both  cases 


Kv) 

pTi 


sup 


Ky) 

X 
(7  H 


(4.3) 


holds,  where  the  sup  is  taken  over  all  available  constants  a  >  0.  Consider  the  process 
Yx  described  by 

Yx(t)  =  x  +  [  b(Yx(s))ds  +  pWH(t).  (4.4) 

Jo 


where  Wh  is  a  fBM  with  the  same  Hurst  parameter  0  <  H  <  1  in  a  (possibly  a 
different)  probability  space  and  Wh  is  adapted  to  a  filtration  (Ft,)-  Then  we 

have  the  following  theorem. 


Theorem  4.1.  Assume  (J-2)  and  (J.3).  Let  Xx  be  any  state  process  which  satisfies 
(4-1).  Let  Yx  be  the  process  described  above  in  (4-4)-  Then, 


P[Xf  reaches  b  before  a]  <  P[YX  reaches  b  before  a}.  (4.5) 


B.  Goal  problems  with  fuel. 

In  [6],  we  also  address  the  above  goal  problem  in  the  presence  of  fuel  available  to 
the  controller.  The  controller  can  spend  the  fuel  to  gain  a  deterministic  displacement 
which  is  proportional  to  the  amount  of  fuel  spent.  This  is  a  singular  control  problem 
driven  by  fBM  noise.  In  this  situation,  we  introduce  two  extreme  strategies  known 
as  bold  play  and  timid  play  where  in  bold  play  strategy,  controller  spends  all  the  fuel 
immediately  and  in  timid  play  strategy,  controller  spends  fuel  only  if  the  position  is  at 
origin  to  receive  a  forward  push.  In  two  theorems,  we  have  established  that  bold  play 
and  timid  play  are  optimal  strategies  under  different  circumstances. 
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5  Utility  maximization. 

We  consider  a  controlled  state  process 


X^(t)  —  x  +  f  u(s)[pi  ds  +  dBH(sj]  (5.1) 

Jo 

where  Wh  is  a  fractional  Brownian  motion  with  Hurst  parameter  \  <  H  <  1  and  is 
adapted  to  a  filtration  (J-t)-  The  control  process  u  is  adapted  to  the  filtration  (JG)  and 
f0  |-u(s)|(is  <  oo.  The  initial  point  x  >  0.  The  stochastic  integral  fQ  u(s)dBH(s )  is  consid¬ 
ered  a  Wick  type  integral  rather  than  the  path-wise  stieltjes  integral  we  have  used  in  the 
comparison  theorems. 

The  problem  addressed  here  is  motivated  by  the  following  continuous-time  gambling  sce¬ 
nario:  Consider  a  gambler  with  initial  wealth  of  x  >  0  dollars  and  who  would  like  to  bet 
an  amount  of  u(t)  dollars  at  time  t  in  a  continuous-time  gambling  scheme  where  the  state 
process  of  the  gamble  is  given  by  /it  +  Buit).  Gambler’s  bet  size  u(t)  at  time  t  may  depend 
on  all  the  information  up  to  time  t.  Hence,  the  gambler’s  fortune  at  time  t  is  given  by  W“(t) 
of  (5.1).  In  this  context,  to  have  the  infinitesimal  mean  rate  of  change  of  fortune  to  be 
/i  E[u(t)]dt,  it  is  natural  to  use  wick-type  stochastic  integral  J*  u(s)dBH(s),  since  wick  type 
integral  satisfies  E  f*  u(s)dBH(s)  =  0. 

Gambler’s  objective  is  to  maximize  the  expected  utility  of  the  terminal  wealth  E[U(X£(t))] 
where  U  is  a  concave  utility  function. 

For  this  problem,  we  derived  an  explicit  optimal  strategy  in  [6]  using  the  fractional  Girsanov 
theorem  and  the  fractional  Clark-Ocone  formula. 


6  Networks  in  heavy  traffic. 

A.  Offered  waiting  times  In  a  recent  manusript  [4],  we  have  analyzed  a  sequence  of 
single-server  queueing  systems  with  impatient  customers  with  general  patience  time 
distributions.  We  established  the  heavy  traffic  approximation  for  the  scaled  offered 
waiting  time  process  and  obtained  a  diffusion  process  as  the  heavy  traffic  limit.  These 
results  lead  to  a  large  class  of  diffusion  processes  with  non-linear  drift  coefficients 
as  such  heavy  traffic  limits.  As  a  consequence,  we  also  obtained  the  same  diffusion 
process  for  the  heavy  traffic  limit  of  the  scaled  queue  length  process.  We  also  showed 
the  convergence  of  the  expected  value  of  an  infinite-horizon  discounted  cost  functional 
of  the  queueing  system  to  that  of  the  diffusion  process  under  the  heavy  traffic  limit. 

B.  Many  server  queueing  systems.  This  project  is  a  joint  work  with  Professor  Avi 
Mandelbaum  who  is  an  expert  in  the  theory  of  queueing  systems  in  heavy  traffic.  Mo¬ 
tivated  by  my  previous  work  [2]  on  queueing  systems  in  heavy  traffic,  we  consider  an 
important  control  problem  which  has  applications  to  management  of  a  large  call  center, 
a  receiver  of  a  wireless  communications  network  or  a  communication  system  related 
to  internet  traffic.  This  work  is  recently  submitted  for  publication  and  the  current 
manuscript  consists  of  52  pages.  We  were  able  to  make  an  important  generalization 
to  currently  available  results  in  the  literature,  where  the  system  manager  is  allowed  to 


choose  a  non-Markovian  blocking  mechanism  which  may  depend  on  the  current  state 
as  well  as  the  past  history  of  the  network. 

We  consider  a  controlled  queueing  system  with  many  servers,  finite-capacity  queue  with 
impatient  customers.  Arrivals  are  general;  impatience  and  service  times  are  exponen¬ 
tial,  and  servers  are  iid.  Customers  who  arrive  to  a  full  queue  are  lost,  and  customers 
who  wait  too  long  in  the  queue  abandon.  Motivated  by  profit  maximization  (or  cost 
minimization)  in  moderate-to-large  communication  networks,  blocking  corresponds  to 
a  busy-signal  and  abandonment  to  a  disconnect  before  being  served.  The  problem  is 
the  tradeoff  between  these  two:  the  larger  the  queue  capacity  the  less  the  blocking 
which,  in  turn,  leads  to  longer  queues,  more  waiting  and  hence  more  abandonment. 
Therefore,  we  intend  to  introduce  linear  costs  proportional  to  the  queue  length,  the 
number  of  blocked  and  abandoning  customers,  and  the  number  of  idle  servers.  These 
costs  are  discounted  over  an  infinite  horizon  to  yield  a  stochastic  control  problem,  in 
which  queue  capacity  is  one’s  control  variable.  We  solve  this  problem  asymptotically,  in 
the  Halfin- Whitt  heavy-traffic  regime.  Specifically,  we  study  cost  minimization  as  the 
arrival  rate,  the  number  of  servers  and  (hence)  the  queue-capacity  increase  indefinitely, 
at  rates  that  give  rise  to  what  has  been  called  the  QED  (Quality-  and  Efficiency-Driven) 
regime:  relatively  few  blocking  and  abandonment  (service-quality)  jointly  with  high 
servers’  utilization  (service-efficiency).  Associated  with  the  original  control  problem, 
there  is  a  diffusion  control  problem.  We  solve  the  latter  explicitly  and  our  solution  led 
to  a  threshold  type  optimal  strategy.  We  use  this  solution  to  construct  asymptotically 
optimal  controls  for  the  controlled  queueing  network  problem. 

For  this,  we  obtained  moment  bounds  for  the  scaled  queue  length.  Then,  we  were 
able  to  derive  an  explicit  asymptotically-optimal  queue-capacity,  including  infinite  ca- 
pacity(no  busy-signals)  if  blocking  costs  far  exceed  abandonment  costs,  ft  turned  out 
that  the  value  function  of  the  diffusion  control  problem  is  a  twice  continuously  dif¬ 
ferentiable  convex  function.  We  show  that  the  value  function  of  the  diffusion  control 
problem  provides  an  achievable  asymptotic  lower  bound  for  the  cost  function  of  the 
original  queueing  problem. 
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